A METHOD AND SYSTEM FOR INVERSION OF DETAIL-IN-CONTEXT 

PRESENTATIONS 

[0001] This application is a continuation of United States Patent Application No. 09/932,088, 
filed August 20, 2001, and incorporated herein by reference, which claims priority from 
5 Canadian Patent Application Nos. 2,328,794 and 2,341,965, filed December 19, 2000 and March 
23, 2001, respectively, and incorporated herein by reference. 

FIELD OF THE INVENTION 

[0002] The invention relates to the field of computer graphics processing. More specifically, the 
invention relates to detail-in-context presentations and the inversion of distortions in detail-in- 
1 0 context presentations. 

BACKGROUND OF THE INVENTION 

[0003] Since the advent of video display terminals as the primary interface to the computer, 
making the best use of the available screen space has been a fundamental issue in user interface 
design. This issue has been referred to as the "screen real estate problem". The necessity for 
15 effective solutions to this problem is growing as the ability to produce and store visual 
information in great volumes is outstripping the rate at which display technology is advancing. 
One solution to the screen real estate problem is the use of detail-in-context presentation 
techniques. Detail-in-context presentations are useful for displaying large amounts of 
information on limited-size computer screens. 

20 [0004] Now, in the detail-in-context discourse, differentiation is often made between the terms 
"representation" and "presentation". A representation is a formal system, or mapping, for 
specifying raw information or data that is stored in a computer or data processing system. For 
example, a digital map of a city is a representation of raw data including street names and the 
relative geographic location of streets and utilities. Such a representation may be displayed 

25 visually on computer screen or printed on paper. On the other hand, a presentation is a spatial 
organization of a given representation that is appropriate for the task at hand. Thus, a 
presentation of a representation organizes such things as the point of view and the relative 
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emphasis of different parts or regions of the representation. For example, a digital map of a city 
may be presented with a work route magnified to reveal street names. Thus, detaiMn-context 
presentations allow for magnification of a particular region of interest (the "focal region") in a 
representation while preserving visibility of the surrounding representation. In other words, in 
5 detail-in-context presentations focal regions are presented with an increased level of detail 
without the removal of contextual information from the original representation. In general, a 
detail-in-context presentation may be considered as a distorted view (or distortion) of a portion 
of the original representation where the distortion is the result of the application of a "lens" like 
distortion function to the original representation. For reference, a detailed review of various 
10 detail-in-context presentation techniques may be found in Carpendale's A Framework for Elastic 
Presentation Space (Carpendale, Marianne S. T., A Framework for Elastic Presentation Space 
(Burnaby, British Columbia: Simon Fraser University, 1999)). 

[0005] One shortcoming of the prior art detail-in-context presentation methods is their inability 
to effectively invert distortions in a detail-in-context presentation back to an original or 
15 undistorted presentation of the representation. The ability to perform such an inversion or inverse 
mapping would be of great value in extending the capabilities of detail-in-context presentations 
to applications such as image editing. For example, the editing of a focal region in a 
representation may be facilitated more easily in a distorted presentation rather than in an 
undistorted presentation. 

20 [0006] The ability to perform an inverse mapping is also necessary for applications involving the 
subsequent distortion of a previously distorted presentation. In other words, inversion would 
allow a presentation system user to accurately position or reposition one or more distortion 
producing "lenses" within a given presentation that has already been distorted. Hence, the 
distorted presentation ultimately viewed by the user may be the end result of a series of distortion 

25 steps wherein the individual distortion steps are not known or are difficult to invert. In fact, the 
need for inversion arises whenever it is necessary to position a lens based on observed 
coordinates in the distorted presentation. This is so because the lens may be directly generated 
only from coordinate information in the undistorted presentation. As such, an inversion is 
necessary to produce the source coordinates for generating the lens. 



2 



[0007] Moreover, inversion provides a means to calculate real distances in an undistorted 
presentation based on locations within one or more lenses in a corresponding distorted 
presentation. For example, if a user wants to know the distance in the undistorted presentation 
between the focal points of two separate lenses in a corresponding distorted presentation of a 
5 map, such as the distance between a current location and a destination location, this distance can 
be computed via inversions of the focal points of these lenses. 

[0008] Several systems are known which provide techniques for converting distorted or warped 
three-dimensional (3D) images into corrected, undistorted, or dewarped two-dimensional (2D) 
images. In United States Patent Number 6,005,611 (Gullichsen, et al.), a system is disclosed 

10 wherein a distorted image captured by a wide-angle or fisheye lens is corrected through the use 
of a specially generated polynomial transform function that maps points from the distorted image 
into rectangular points. A more complex transform function is described in United States Patent 
Number 5,185,667 (Zimmerman). In United States Patent Number 5,329,310 (Liljegern, et al.) a 
similar objective is achieved in the context of motion picture images through the use of multiple 

15 lens (camera and projector) transfer functions. The result being the ability to project an image, 
taken from a particular point of view, onto a screen, especially a curved wide angle screen, from 
a different point of view, to be viewed from the original point of view, without distortion. In 
United States Patent Number 5,175,808 (Sayre), a method and apparatus for non-affine image 
warping is disclosed that uses displacement tables to represent the movement of each pixel from 

20 an original location in a source image to a new location in a warped destination image. Through 
these displacement tables and a resampling method, the need for inversion of the underlying 
transform equation that specify the distortion or warp is eliminated. Finally, in United States 
Patent Number 4,985,849 (Hideaki), look-up tables are used in combination with the forward 
evaluation of the transform equation in order to avoid the step of transform equation inversion. 

25 However, none of these systems disclose a method and system for inverting distortions in a 
manner that is optimized for detail-in-context presentations. 

[0009] A need therefore exists for a method and system that will allow for the effective inversion 
of distortions in detail-in-context presentations. Therefore, it is an object of the present invention 
to obviate or mitigate at least some of the above mentioned disadvantages. 
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SUMMARY OF THE INVENTION 



[0010] The invention provides a method and system for the inversion of distortions in detail-in- 
context presentations. According to one aspect of the invention, a method is provided that allows 
a distortion in a detail-in-context presentation to be inverted. The method comprises the steps of 
5 locating a first approximation point in an undistorted surface for the inversion of a point in a 
distorted surface, determining if the approximation point is acceptable as an inversion of the 
point in the distorted surface, locating a next approximation point in the undistorted surface if the 
first approximation point is not acceptable, and repeating this process until an acceptable 
approximation point is located for the inversion of the point in the distorted surface. According 

10 to another aspect of the invention, the use of this method to obtain the distance between points on 
an undistorted surface from the relative distances between corresponding points on a plurality of 
distorted surfaces in a detail-in-context presentation is provided. According to another aspect of 
the invention, a data processing system is provided. This data processing system has stored 
therein data representing sequences of instructions which when executed cause the above- 

15 described method to be performed. The data processing system generally has an input device, a 
central processing unit, memory, and a display. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0011] The invention may best be understood by referring to the following description and 
accompanying drawings which illustrate the invention. In the drawings: 

20 [0012] FIG. 1 is a cross-sectional view of a presentation illustrating a point X on a distorted 
surface and a first approximation point P 0 for its inversion back to an original basal plane in 
accordance with the preferred embodiment; 

[0013] FIG. 2 is a cross-sectional view of a presentation illustrating the displacement of a first 
approximation point P 0 onto a distorted surface by application of a distortion function D 
25 resulting in a point P 0 D in accordance with the preferred embodiment; 

[0014] FIG. 3 is a cross-sectional view of a presentation illustrating the projection of a point P 0 D 
onto a line RVP-X resulting in a point P 0 P in accordance with the preferred embodiment; 
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[0015] FIG. 4 is a cross-sectional view of a presentation illustrating the projection of a point P 0 P 
onto a basal plane resulting in a second approximation point Pi and a corresponding displaced 
point Pi D in accordance with the preferred embodiment; 

[0016] FIG. 5 is a cross-sectional view of a presentation illustrating the projection of a point Pi D 
5 onto a line RVP-X resulting in a point Pi P which is then projected onto a basal plane resulting in 
a third approximation point P2 in accordance with the preferred embodiment; 

[0017] FIG. 6 is a block diagram of a data processing system in accordance with the preferred 
embodiment; 

[0018] FIG. 7 is a flow chart illustrating an iterative method for inversion in accordance with the 
1 0 preferred embodiment. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0019] In the following description, numerous specific details are set forth to provide a thorough 
understanding of the invention. However, it is understood that the invention may be practiced 
without these specific details. In other instances, well-known software, circuits, structures and 

1 5 techniques have not been described or shown in detail in order not to obscure the invention. The 
term data processing system is used herein to refer to any machine for processing data, including 
the computer systems and network arrangements described herein. The term "Elastic 
Presentation Space" or "EPS" is used herein to refer to techniques that allow for the adjustment 
of a visual presentation without interfering with the information content of the representation. 

20 The adjective "elastic" is included in the term as it implies the capability of stretching and 
deformation and subsequent return to an original shape. EPS graphics technology is described by 
Carpendale in A Framework for Elastic Presentation Space (Carpendale, Marianne S. T., A 
Framework for Elastic Presentation Space (Burnaby, British Columbia: Simon Fraser 
University, 1999)) which is incorporated herein by reference. Basically, in EPS graphics 

25 technology, a two-dimensional visual representation is placed onto a surface; this surface is 
placed in three-dimensional space; the surface, containing the representation, is viewed through 
perspective projection; and the surface is manipulated to effect the reorganization of image 
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details. The presentation transformation is separated into two steps: surface manipulation or 
distortion and perspective projection. 

[0020] In general, the invention described herein provides a method and system for the inversion 
of distortions in detail-in-context presentations. The method and system described is applicable 
5 to detail-in-context navigation within computer graphics processing systems including EPS 
graphics technology and to computer graphics processing systems in general. 

[0021] According to one aspect of the invention, a method is described that allows a distortion in 
a detail-in-context presentation to be inverted. The method comprises the steps of locating a first 
approximation point in an undistorted surface for the inversion of a point in a distorted surface, 
10 determining if the approximation point is acceptable as an inversion of the point in the distorted 
surface, locating a next approximation point in the undistorted surface if the first approximation 
point is not acceptable, and repeating this process until an acceptable approximation point is 
located for the inversion of the point in the distorted surface. 

[0022] According to another aspect of the invention, the use of this method to obtain the distance 
15 between points on an undistorted surface from the relative distances between corresponding 
points on a plurality of distorted surfaces in a detail-in-context presentation is described. 

[0023] According to another aspect of the invention, a data processing system is described. This 
data processing system has stored therein data representing sequences of instructions which 
when executed cause the above-described method to be performed. The data processing system 
20 generally has an input device, a central processing unit, memory device, and a display. 

[0024] Referring to FIG. 6, there is shown a block diagram of an exemplary data processing 
system 600 according to one embodiment of the invention. The data processing system is 
suitable for implementing EPS graphics technology. The data processing system 600 includes an 
input device 610, a central processing unit or CPU 620, memory 630, and a display 640. The 
25 input device 610 may be a keyboard, mouse, trackball, or similar device. The CPU 620 may 
include dedicated coprocessors and memory devices. The memory 630 may include RAM, 
ROM, databases, or disk devices. And, the display 640 may include a computer screen or 
terminal device. The data processing system 600 has stored therein data representing sequences 
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of instructions which when executed cause the method described herein to be performed. Of 
course, the data processing system 600 may contain additional software and hardware a 
description of which is not necessary for understanding the invention. 

[0025] Referring to FIGURES 1 through 7 the method of one embodiment of the invention will 
5 now be described. With this method, a point in an undistorted presentation or data space is 
found, which when distorted, yields a specified point in a distorted presentation or data space. 
Then, if desired, the inversion of the entire distorted presentation or data space to an original 
undistorted presentation or data space may be obtained as the inverse mapping of the locus of 
points in the distorted presentation or data space. The method is iterative and makes use of the 
10 distortion process itself as a component in an approximation technique for computing the inverse 
of the distortion. 

[0026] Referring to FIG. 1, there is shown a cross-sectional view of a presentation 100 in 
accordance with EPS graphics technology and in accordance with the preferred embodiment. 
EPS graphics technology employs viewer-aligned perspective projections to produce detail-in- 

15 context presentations in a reference view plane 101 which may be viewed on a display 640. 
Undistorted two-dimensional (2D) data points are located in a basal plane 110 of a three- 
dimensional (3D) perspective viewing volume 120 which is defined by extreme rays 121 and 122 
and the basal plane 110. A reference viewpoint (RVP) 140 is located above the centre point of 
the basal plane 110 and reference view plane 101. Points in the basal plane 110 are displaced 

20 upward onto a distorted surface 130 which is defined by a general three-dimensional distortion 
function D. The direction of the viewer-aligned perspective projection corresponding to the 
distorted surface 130 is indicated by the line F 0 - F 131 drawn from a point F 0 132 in the basal 
plane 110 through the point F 133 which corresponds to the focus or focal region of the distorted 
surface 130. The method of the present invention locates a point Pj in the basal plane 110 that 

25 corresponds to a point X 150 on the distorted surface 130 through a series of steps, involving 
iteration and approximation, as follows. Successive approximations of the point Pi are 
represented by the subscript i where i >0. 

[0027] Referring to FIG. 7, where there is shown a flow chart 700 illustrating the method of one 
embodiment of the invention, and again referring to FIG. 1, at step 1, an acceptable tolerance 6 is 
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selected for the magnitude of the difference between the point X 150 on the distorted surface 
130 and the point Pj D , where Pj D represents the result of the mapping of a point Pi in the basal 
plane 110 onto the distorted surface 130 through the function D. The value of 5 is application 
dependent. For example, an acceptable 5 could be less than half the width of a pixel for a typical 
5 display surface such as a monitor 640. In general, successive approximations of the point Pi will 
continue until the magnitude of the difference between the point X 150 and the point Pj 0 is less 
than 8, that is, until | Pj D - X | < 8. 

[0028] At step 2, a first approximation point P 0 160 for the inversion of X 150 is located at the 
intersection point in the basal plane 110 of a line RVP-X 170 drawn through RVP 140, X 150, 
1 0 and the basal plane 110. Here, i = 0. 

[0029] Referring to FIG. 2, at step 3, point P 0 160 is displaced onto the distorted surface 130 by 
the application of D. The resultant point on the distorted surface 130 is represented by P 0 D 210. 

[0030] At step 4, the magnitude of the difference between the point X 150 and the point P 0 D 210 
is calculated. If | P 0 D - X | < 8, then an acceptable value for the inversion of the point X 150 
15 will have been found and the method is complete for the point X 150. The method may then 
proceed with the inversion of another point on the distorted surface 130. If | P 0 D - X | > 8, then 
the method will continue and will generate a next approximation for the inversion of the point X 
150. 

[0031] Referring to FIG. 3, at step 5, the point P o D 210 is projected onto the line RVP-X 170 to 
20 locate the point P 0 P 310 which is the closest point to P 0 D 210 on the line RVP-X 170. 

[0032] Referring to FIG. 4, at step 6, the point P o P 310 is projected onto the basal plane 110 to 
produce the next approximation Pi 410 for the inversion of the point X 150. This projection is 
made in the direction parallel to a line F - F 0 , that is, in the direction from point F 133 to point F 0 
132 parallel to the line F 0 - F 131. Alternately, this direction can also be established by applying 
25 the distortion function D to any point in the basal plane within the lens extent (as defined by the 
distortion function D). The direction of the displacement of such a point by the distortion 
function D will be antiparallel to the line F - F 0 . The point Pi 410 is thus located on the basal 
plane 110 at the point of intersection of the basal plane 110 and a line 420 drawn parallel to the 
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line F 0 - F 131 and passing through the point P 0 P 310. Now, i = 1 and a second iteration may 
begin from step 3. 

[0033] Referring to FIG. 5, a second iteration is illustrated resulting in point P 2 510 as an 
approximation of the inversion of the point X 150. 

5 [0034] Again referring to FIG. 1, in certain cases such as folding, which is the lateral 
displacement of a focal region 133 through shearing of the viewer-aligned vector defining the 
direction of distortion 131, it is possible for successive approximations for Pj to diverge. This 
may be caused, for example, by a fold in which a undistorted area of the basal plane 110 is 
hidden by a portion of the distorted surface when viewed from RVP 140 such that a line drawn 
10 through RVP 140, the distorted surface, and the basal plane 110 intersects the distorted surface at 
multiple points. In these circumstances, a bisection of approximation points Pj may be used to 
search for the desired intersection of RVP-X 170 with the basal plane 110. 

[0035] The method of the embodiment of the invention described above may be used to obtain 
the distance between points on an undistorted surface from the relative distances between 

15 corresponding points on one or more distorted surfaces in a detail-in-context presentation. For 
example, if one point is selected on a first distorted surface and a second point is selected on a 
second distorted surface, both surfaces being contained in a detail-in-context presentation, then 
the distance between these two points on the undistorted surface may be found by first inverting 
each point on each distorted surface, using the method of the invention, to obtain corresponding 

20 points on the undistorted surface. Then, the required distance may be calculated as the magnitude 
of the difference between the two inverted points. 

[0036] To reiterate and expand, the method and system of the present invention includes the 
following unique features and advantages: it facilitates the location of a point in an undistorted 
presentation which, when distorted, yields a specified point in a distorted presentation and then, 
25 from this point, the inversion of the entire distorted presentation back to the original undistorted 
presentation may be accomplished through the inverse mapping of the locus of the points in the 
distorted presentation; it employs an iterative approach to inversion to facilitate general 
distortion functions; in other words, having knowledge of the location of the point to be inverted 
in the distorted presentation and through an iterative process, it computes a series of points in the 
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undistorted presentation space until a point is found, which when displaced by a general 
distortion function, yields a point that is coincident with the point to be inverted in the distorted 
presentation; it is not specific to a particular distortion function or transform equation; it does not 
require the maintenance of a copy of the undistorted presentation in computer memory; it does 
not use look-up tables and hence does not put unacceptable demands on computing system 
resources, including memory, especially for undistorted presentations that are large in size; and, 
it may be used to obtain the distance between points in an undistorted presentation from the 
relative distances between corresponding points on one or more distorted surfaces in a detail-in- 
context presentation. 

[0037] Although the invention has been described with reference to certain specific 
embodiments, various modifications thereof will be apparent to those skilled in the art without 
departing from the spirit and scope of the invention as outlined in the claims appended hereto. 
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